Skip to content

Feat: Generator fan-out in transform pipeline#59

Merged
bpolaszek merged 3 commits intomasterfrom
feat/generator-fan-out
Feb 11, 2026
Merged

Feat: Generator fan-out in transform pipeline#59
bpolaszek merged 3 commits intomasterfrom
feat/generator-fan-out

Conversation

@bpolaszek
Copy link
Owner

Summary

  • ChainTransformer: Generators are now consumed between each step of the chain. Each yielded item is passed individually to the next transformer (flatMap semantics), instead of passing the Generator object itself.
  • EtlExecutor: A TransformEvent is now dispatched per transformed item instead of once for the entire batch. SkipRequest in a listener only skips the individual item, not all items from the generator.
  • Scalar/array return values are unaffected — only Generator instances trigger fan-out.

Breaking changes

Impact Description
⚠️ ChainTransformer Chained transformers that type-hint Generator $items will break — they now receive individual items instead
⚠️ TransformEvent listeners $event->transformResult now contains a single item instead of a batch. [...$e->transformResult] returns ['item'] instead of ['item1', 'item2']
⚠️ Skip in TransformEvent skip() now only skips the individual transformed item, not all items from the generator
ℹ️ LoggerRecipe onTransform is called N times instead of 1 for a generator of N items (same currentItemKey). More verbose but more informative. No code change needed

Example

$etl = (new EtlExecutor())
    ->transformWith(
        fn (string $item): string => strrev($item),
        function (string $item): Generator {
            yield $item;
            yield strtoupper($item);
        },
        fn (string $item): string => "({$item})",
    );

$report = $etl->process(['foo', 'bar']);
// Output: ['(oof)', '(OOF)', '(rab)', '(RAB)']

Test plan

  • All 129 tests pass (237 assertions)
  • PHPStan clean
  • 100% code coverage maintained

🤖 Generated with Claude Code

Generators returned by transformers are now consumed between each step
of a ChainTransformer, passing individual items to the next transformer.
TransformEvents are dispatched per-item instead of per-batch.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@codecov
Copy link

codecov bot commented Feb 10, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (86de4e7) to head (a51e760).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff             @@
##              master       #59   +/-   ##
===========================================
  Coverage     100.00%   100.00%           
- Complexity       347       352    +5     
===========================================
  Files             61        61           
  Lines            897       916   +19     
===========================================
+ Hits             897       916   +19     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@bpolaszek
Copy link
Owner Author

@junie-agent review

@github-actions
Copy link

github-actions bot commented Feb 10, 2026

Junie is failed!

Details: ❌ Junie output file path is not set.

This indicates that Junie execution did not complete properly.
Please check the Junie execution logs above for error details.

View job run

@bpolaszek bpolaszek merged commit 26fdf50 into master Feb 11, 2026
6 checks passed
@bpolaszek bpolaszek deleted the feat/generator-fan-out branch February 11, 2026 13:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant